In order to solve the problem that image details and small target information are lost prematurely in the subsampling operations of Deeplab V3 Plus， an adaptive attention mechanism image semantic segmentation algorithm based on Deeplab V3 Plus network architecture was proposed. Firstly， attention mechanism modules were embedded in the input layer， middle layer and output layer of Deeplab V3 Plus backbone network， and a weight value was introduced to be multiplied with each attention mechanism module to achieve the purpose of constraining the attention mechanism modules. Secondly， the Deeplab V3 Plus embedded with the attention modules was trained on the PASCAL VOC2012 common segmentation dataset to obtain the weight values （empirical values） of the attention mechanism modules manually. Then， various fusion methods of attention mechanism modules in the input layer， the middle layer and the output layer were explored. Finally， the weight value of the attention mechanism module was automatically updated by back propagation and the optimal weight value and optimal segmentation model of the attention mechanism module were obtained. Experimental results show that， compared with the original Deeplab V3 Plus network structure， the Deeplab V3 Plus network structure with adaptive attention mechanism has the Mean Intersection over Union （MIOU） increased by 1.4 percentage points and 0.7 percentage points on the PASCAL VOC2012 common segmentation dataset and the plant pest dataset， respectively.